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q/ Abstract 

Mh ' In dealing with asymptotic approximation of possibly divergent nets of probability 

C^ , distributions, we are led to study uniform structures on the set of distributions. This 

paper identifies a class of such uniform structures that may be considered to be reasonable 
generalizations of the weak topology. It is shown that all structures in the class yield 
the same notion of asymptotic approximation for sequences (but not for general nets) of 
probability distributions. 
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^ ■ 1 Introduction 

Traditionally limit theorems in probability theory are formulated in terms of the weak topology 

r | on probability distributions, also known as the topology of convergence in distribution. When 

a sequence of probability distributions converges in the weak topology, its limit serves as a weak 
approximation of the distributions in the sequence. 

In more general limit theorems, a sequence of probability distributions, not necessarily 
convergent, is asymptotically approximated by another sequence. That leads to a natural 
question: Are there essentially different "reasonable" notions of asymptotic approximation 
that all reduce to the weak approximation in the special case of convergent sequences? 

To deal with approximation for possibly divergent sequences we need more than a mere 
topology. Asymptotic approximation is conveniently formulated in terms of uniform structures 
(uniformities) on the space of probability distributions. The question above then becomes: 
What "reasonable" uniform structures on the space of probability distributions are compatible 
with the weak topology? 

Uniform structures on probability distributions on a metric space were investigated by 
Dudley [B][5J 11.7] and DAristotile, Diaconis and Freedman (3J. Davydov and Rotar [3] showed 
that the previously neglected uniformity defined by bounded uniformly continuous functions is 
a reasonable uniformity in the above mentioned sense. 
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Uniform structures on probability distributions are also implicit in the rich literature on 
probability metrics, surveyed by Dudley [7], Gibbs and Su [10] , Rachev [16] and Zolotarev [19j ; 
however, so far the focus in that area has been on quantitative results for convergent sequences, 
not on general uniform structures or even the uniform structures defined by probability metrics. 

In this paper I offer further evidence that the uniformity proposed by Davydov and Rotar 
is an appropriate analog of the weak topology, at least when we deal with sequences (rather 
than general nets) of probability distributions. To formalize the notion of "reasonable" uniform 
structures, I formulate two natural properties that they should satisfy. Although there is usually 
more than one uniformity satisfying those properties, and therefore more than one notion of 
a "weak" asymptotic approximation for nets of probability distributions, they all agree on 
sequences. 

Version 1 of the paper was dated July 20, 2010. Version 2 incorporates a suggestion from 
Ramon van Handel, which considerably simplifies and improves the results in section [31 

2 Preliminaries 

Uniform structures (uniformities) and uniform spaces may be defined in several equivalent ways. 
For the purposes of this paper, the most suitable definition is the one based on pseudomet- 
rics [TTJ Ch.15], although definitions in terms of entourages [1] or uniform covers [12] would 
do as well. Let U be a uniform structure on a set A. A net {a 7 } 7 in A is an asymptotic 
1A- approximation of a net {fr 7 } 7 in A iff lim 7 A(a 7 , 6 7 ) = for every Z-/-uniformly continuous 
pseudometric A. Here a net is a family indexed by a directed partially ordered set. A sequence 
is a family indexed by the totally ordered set {0, 1,2,...}. 

The set of real numbers is denoted R. When S a metric space, Cb(S) is the space of bounded 
real- valued continuous functions on S, and Ub(S') is the space of bounded real- valued uniformly 
continuous functions on S. The sup norm of a function /eCb(S') is ||/||s : = suPzesl/C 2 -)!- 

The Borel a-algebra Bo(S) is the smallest cr-algebra of subsets of S containing all open sets 
in S. In this paper, a measure on S means a bounded signed measure on 60(5*). A measure fi 
on S is tight if 

^|(A) = sup{|^|(C) I C is compact and CCA} 

for every A 6 Bo(S*). Tight measures are also known as Radon or regular measures. The space 
of tight measures on S is denoted M t (5), and 

Pt(5) := {MGM t (5) I \x > and n(S) = 1} 

is the space of tight probability measures on S. The point mass at xGS is denoted ds(x); that 
defines a mapping ds'- S — > Pt(S). 

The vector space duality (M t (5),Cb(5)) is defined by integration: 



(Hj):=jfdli for n€M t (S),f€C b (S). 



Since every /ie M t (5) is uniquely determined by the values (/i, /), /eUb(S'), integration defines 
also a vector space duality (M t (5), Ub^)). When X is a real-valued random variable, E(JQ 
denotes the expected value of X. If / e Cb(S') and X is an S'-valued random variable with 
distribution LK=P t (S) then (/i, f) = E(/(X)). 

When 5 and S" are metric spaces and ip: S — > S" is a continuous mapping, the linear 
mapping M t (<£>): M t (5) ->• M t (S") is defined by (M t (^)(^),#) := (fi 7 g o ip) for /i 6 M t (5) and 
.geC b (S"). Clearly M t (<p) maps P t (5) into P t (S"). 

When d is the metric of S and /16 M t (5), write 

BLip„(d):={/:S->R|||/||s<l and \f(x)-f(y)\<d(x,y) for all x,yeS} 
Nl* := sup {</*,/> | /GBLip b (d)}. 

By the following well-known lemma, ||-||* is a norm on the space Mt(S'). This norm is equivalent 
to the norm ||-||b L studied by Dudley [5]|B]. 

Lemma 1 Let S be a metric space with metric d. The space UnLj. nBL\p^(d) is \\-\\s-dense in 
the space \Jt,(S). 

Proof. Take any /GUb(S'), e > 0. There is 8 > such that i£x,y£S and d(x,y) < 8 then 
|/(x) — /(y)| < e. Choose an integer n > max(||/||s + e,2\\f\\s/8) and define 

g(y) :=sup (f(x)-e-nd(x,y)) for ye 5. 

xes 

Then ff GnBLip b (d) and f-e <g < f. D 

In addition to the ||-||* topology, consider also two weak topologies on M t (5): 

• The Cb(S')-weak topology from the duality (M t (5), Cb(S)). That is, the topology of simple 
convergence on the elements of Cb(S). 

• The Ub(S')-weak topology from the duality (M t (5), Ub(S')). That is, the topology of simple 
convergence on the elements of Ub(5). 

The restriction of the Cb(5*)-weak topology to Pt('S') is called simply the weak topology (or the 
topology of convergence in distribution) in probability theory. 

Lemma 2 Let S be a metric space. The Ct,(S)-weak topology, the \Jt,(S)-weak topology and the 
||'||* topology coincide on P t (S). 

Proof. LeCam [T31 Lem.5] proves that the two weak topologies coincide on Pt(S')- The 
equivalence with the ||-||* topology is proved by LeCam [14], and by Dudley [5J Th.18]. D 

The proof of Corollary [TT] in section [3] relies on the following theorem, which generalizes the 
Schur property 5.19] of convergent sequences in l-y. 



Theorem 3 Let S be a metric space. The \Jb{S)-weak topology and the \\-\\* topology on M t (5) 
have the same compact sets, and therefore the same convergent sequences. 

Theorem [3] appeared in [13]. A simpler proof was found by Cooper and Schachermayer [2]. 
Variants of the theorem were proved by van Handel [18j B.l] (for measures on R") and Davydov 
andRotar 4, Th. 4]. 

Theorem 4 Let S be a metric space and let fXj, Vj £ Pt(5Q for j — 0,1, ... . We have 

lim || \Xj — Vj || * = 

j 

if and only if there exist S -valued random variables Xj, Yj such that the distribution of Xj is 
/ij, the distribution ofYj is Vj, and lirrij d(Xj,Yj) — almost surely. 
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Theorem [4] is proved by Dudley [8i 11.7.1]. (In [8], j3 denotes the metric of the norm 
which is equivalent to ||-||*.) 

When E is a locally convex vector space, the topology of E is defined by the family of 
continuous seminorms [171 II. 4]. The additive uniformity on any subset of E is the uniformity 
induced by the pseudometrics of the form (x, y) h-> s(x — y), x,y£E, where s is a continuous 
seminorm on E. Each of the three topologies on M t (5) defined above makes M t (5) into a locally 
convex space. The corresponding additive uniformities on subsets of M t (S f ) (in particular, on 
Pt(S')) will be referred to as the ||-||* uniformity, the Cb(5')-weak uniformity and the Ub(S')-weak 
uniformity. 

Lemma 5 Let S be a metric space. On P t (S), the \\-\\* uniformity is finer than the \J^(S)-weak 
uniformity. 

Proof. Follows from Lemma Q] □ 



3 Uniform structures on Pt(S) 



In this section I describe a class of uniform structures on Pt{S) that appear to be reasonable 
candidates for extending to divergent nets the notion of weak approximation on convergent 
nets. Of course, it is highly subjective and context-dependent what uniformities should be 
considered "reasonable" for this purpose. I replace that subjective notion by the two properties 
in Definition [7] 

Lemma 6 Let I n := [— n, n] C R with the standard metric. There is a unique uniformity on 
Pt(In) compatible with the Ct,(I n )-weak topology. This unique uniformity coincides with the \\-\\* 
uniformity, the Cb{I n )-weak uniformity and the Ub{I n )-weak uniformity on P t {I n )- 



Proof. Since /„ is compact, so is Pt(I n ) with the Cb(/ ra )-weak topology. It follows that 
there is only one uniformity on Pt{I n ) compatible with the Cb(/ n )-weak topology [HI 11.24], 
and by Lemma [Hit coincides with the three uniformities listed. □ 

Property (Al) in the next definition and the resulting simplification of the proofs that follow 
were suggested by R. van Handel. 

Definition 7 Let S be a metric space with metric d. Consider the following properties of 
a uniform structure U on Pt(<5). 

(Al) If Xj and Yj, j — 0, 1, ... , are S-valued random variables with distributions \Xj G Pt(S') 
and Va G Pt(S') respectively and i/hm,- d(Xj, Yj) = almost surely then the sequence {fij}j 
is an asymptotic U- approximation for the sequence {fj}j- 

(A2) For every interval I n '■— [—n, n] C R, n = 1,2, . . . , if a mapping <p: S — > I„ is uniformly 
continuous then so is the mapping Mt(ip) from Pt(S) with U to Pt (in) with the unique 
uniformity in Lemma\Q 

Property (Al) states that asymptotic approximation (understood almost surely) for se- 
quences of S- valued random variables implies asymptotic approximation for the corresponding 
distributions. 

(A2) is a functorial property of the assignment X M> (P t (S , ),W) for uniformly continuous 
mappings tp: S —$■ I n , assuming that on convergent nets in Pt(/ n ) the asymptotic approximation 
agrees with the weak approximation. 

As a special case of (Al), if {xj}j and {yj}j are two sequences of points in S such that 
linij d(xj, yj) — then {d(xj)}j is an asymptotic ^/-approximation for {d(yj)}j. Consider the 
points Xj := j and yj = j + 1/j, j = 1,2,..., in R. There is a function / 6 Cb(R) such that 
f(xj) — f(yj) = 1 for all j; it follows that the Cb(M)-weak uniformity on P t (M) does not satisfy 
(Al). 

The other two uniformities from section [5] have properties (Al) and (A2): 

Theorem 8 For every metric space S the (Jb(S)-weak uniformity and the ||-||* uniformity on 
Pt(S) have properties (Al) and (A2). 

Proof. Let /ij, Vj s Pt(A), j = 0,1, . . . , be the distributions of S-valued random variables 
Xj, Yj such that linij d(Xj, Yj) = almost surely. Then 

\fa,f) - (Uj,f)\ = \E(f(Xj)) - E(f(Yj))\ <E(\f(Xj) f(Yj)\) 

11^-^11*= sup \{»j,f)-(vj,f)\<E(2Ad(Xj,Yj)) 
/eBLip b (d) 

and if \im.jd(Xj,Yj) — almost surely then linx, ||/Xj — Vj\\* — 0. Hence the ||-||* uniformity 
has property (Al), and so does the Ub(A)-weak uniformity by LemmaO 

(A2) follows from the definition of M t (ip) and Lemma [6] D 



Next I prove that every uniformity U satisfying (Al) and (A2) is between the two unifor- 
mities in Theorem [8) 

Theorem 9 Let X be a metric space with metric d, andlA a uniform structure on P t (A). 

1. If 14 has property (Al) then it is coarser than the \\-\\* uniformity on P t (A). 

2. If U has property (A2) then it is finer than the \Jt,(X)-weak uniformity on P t (A). 

Proof. To prove part 1, assume that U is not coarser than the ||-||* uniformity on Pt(A). 
That means that there are a W-uniformly continuous pseudometric A, e > and jj,j, Vj <E Pt(A) 
for j = 0, 1, . . . such that linx, \\/ij — Vj\\* =0 and A(/Xj, Vj) > £ for all j. By Theorem @] there 
are S- valued random variables Xj , Yj whose distributions are \ij , Vj respectively and such that 
linij d(Xj, Yj) — almost surely. Thus U does not have property (Al). 

2. Assume that U has property (A2) and take any / e Ub(A). Choose n G {1, 2, . . . } for 
which \\f\\x < n, so that / maps X into I n :— [—n,n]. 

Let g £ Ub(In) be the function g : x i— > x. By (A2) and Lemma [6] the mapping M t (/) is 
uniformly continuous from Pt(A) with U to Pt(I n ) with the Ub(/ n )-weak uniformity. Thus the 
mapping 

M^(Mt(/)(/x),5) = {M,5°/> =(M,/> 

from Pt(A) with U to M is uniformly continuous. That proves that U is finer than the Ub(A)- 
weak uniformity. □ 



Corollary 10 Let X be a metric space with metric d, andU a uniform structure on P t (X). If 
IA has properties (Al) and (A2) then it is compatible with the Ct > (X)-weak topology on P t (X). 

Proof. Apply Theorem |H] and Lemma [S] □ 



Corollary 11 Let S be a metric space andlA a uniform structure on P t (S). IflA has properties 
(Al) and (A2) then the following statements are equivalent for any two sequences {hj}j and 
{Uj}j in P t (5). 

(i) linij ((Hj,f) - (vj,f)) = for every fe\J b (S). 

(ii) The sequence {(J-j}j is an asymptotic U- approximation of the sequence \yj\j ■ 

(in) hmj \\fij -vj\\* = 0. 

(iv) There exist S-valued random variables Xj, Yj, j — 0,1,..., whose distributions are fij 
and Vj and such that linx, d(Xj, Yj) — almost surely. 



Proof. The implication (i)=^(iii) follows from Theorem [U (iii)=>(ii)=>(i) from Theorem [§J 
and (iii)'O-(iv) from Theorem |4j D 

The equivalence of (i) and (iii) in the corollary was derived by Davydov and Rotar [3] from 
their variant of Theorem [3] By [SJ 11.7.1], condition (iii) is also equivalent to each of the 
following: 

• linij p(/ij, Vj) — 0, where p is the Levy-Prokhorov metric. 

• There exist S- valued random variables Xj, Yj, j = 0, 1, ... , whose distributions are /ij 
and Vj and such that linx,- d(Xj , Yj ) = in probability. 

4 Concluding remarks 

By Corollary [TTJ all uniformities on Pt(S') that satisfy (Al) and (A2) yield the same notion of 
asymptotic approximation for sequences of tight probability measures, namely the approxima- 
tion defined by the Ub(S')-weak and ||-||* uniformities. 

When we look beyond sequences and deal with general nets of probability distributions, 
there are multiple notions of asymptotic approximation, even for the uniformities with prop- 
erties (Al) and (A2). For example, the ||-||* uniformity on P t (K) is strictly finer than the 
Ub(K)-weak uniformity, and thus there are two nets {/i 7 } 7 and {^ 7 } 7 in Pt(R) such that {^ 7 } 7 
is an asymptotic Ut>(R)-weak approximation of {a* 7 } 7 but not an asymptotic ||-||* approxima- 
tion. 
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